Human-Robot Gym: Benchmarking Reinforcement Learning in Human-Robot Collaboration

Deep reinforcement learning (RL) has shown promising results in robot motion planning with first attempts in human-robot collaboration (HRC). However, a fair comparison of RL approaches in HRC under the constraint of guaranteed safety is yet to be made. We, therefore, present human-robot gym, a benchmark for safe RL in HRC. Our benchmark provides eight challenging, realistic HRC tasks in a modular simulation framework. Most importantly, human-robot gym includes a safety shield that provably guarantees human safety. We are, thereby, the first to provide a benchmark to train RL agents that adhere to the safety specifications of real-world HRC. This bridges a critical gap between theoretic RL research and its real-world deployment. Our evaluation of six environments led to three key results: (a) the diverse nature of the tasks offered by human-robot gym creates a challenging benchmark for state-of-the-art RL methods, (b) incorporating expert knowledge in the RL training in the form of an action-based reward can outperform the expert, and (c) our agents negligibly overfit to training data

Reducing Safety Interventions in Provably Safe Reinforcement Learning

Deep Reinforcement Learning (RL) has shown promise in addressing complex robotic challenges. In real-world applications, RL is often accompanied by failsafe controllers as a last resort to avoid catastrophic events. While necessary for safety, these interventions can result in undesirable behaviors, such as abrupt braking or aggressive steering. This paper proposes two safety intervention reduction methods: proactive replacement and proactive projection, which change the action of the agent if it leads to a potential failsafe intervention. These approaches are compared to state-of-the-art constrained RL on the OpenAI safety gym benchmark and a human-robot collaboration task. Our study demonstrates that the combination of our method with provably safe RL leads to high-performing policies with zero safety violations and a low number of failsafe interventions. Our versatile method can be applied to a wide range of real-world robotic tasks, while effectively improving safety without sacrificing task performance.Comment: 8 pages, 6 figure

A novel approach to label road defects in video data: semi-automated video analysis

Road defects like potholes have a major impact on road safety and comfort. Detecting these defects manually is a highly time consuming and expensive task. Previous approaches to detect road events automatically using acceleration sensors and gyro meters showed good results. However, these results could be significantly improved with additional usage of image analysis. A large, labeled image data set is required for training and validation. This paper presents a method to automate parts of the labeling task. The method is based on a simple two step approach: at first, an unsupervised algorithm detects possible events based on the acceleration data and filters those video sequences with defects. Second, a human operator decides based on the short video sequences if the event was due to an existing road defect and labels the corresponding area in an image

Characterization of Road Condition with Data Mining Based on Measured Kinematic Vehicle Parameters

This work aims at classifying the road condition with data mining methods using simple acceleration sensors and gyroscopes installed in vehicles. Two classifiers are developed with a support vector machine (SVM) to distinguish between different types of road surfaces, such as asphalt and concrete, and obstacles, such as potholes or railway crossings. From the sensor signals, frequency-based features are extracted, evaluated automatically with MANOVA. The selected features and their meaning to predict the classes are discussed. The best features are used for designing the classifiers. Finally, the methods, which are developed and applied in this work, are implemented in a Matlab toolbox with a graphical user interface. The toolbox visualizes the classification results on maps, thus enabling manual verification of the results. The accuracy of the cross-validation of classifying obstacles yields 81.0% on average and of classifying road material 96.1% on average. The results are discussed on a comprehensive exemplary data set

Learning from the crowd: Road infrastructure monitoring system

The condition of the road infrastructure has severe impacts on the road safety, driving comfort, and on the rolling resistance. Therefore, the road infrastructure must be monitored comprehensively and in regular intervals to identify damaged road segments and road hazards. Methods have been developed to comprehensively and automatically digitize the road infrastructure and estimate the road quality, which are based on vehicle sensors and a supervised machine learning classification. Since different types of vehicles have various suspension systems with different response functions, one classifier cannot be taken over to other vehicles. Usually, a high amount of time is needed to acquire training data for each individual vehicle and classifier. To address this problem, the methods to collect training data automatically for new vehicles based on the comparison of trajectories of untrained and trained vehicles have been developed. The results show that the method based on a k-dimensional tree and Euclidean distance performs best and is robust in transferring the information of the road surface from one vehicle to another. Furthermore, this method offers the possibility to merge the output and road infrastructure information from multiple vehicles to enable a more robust and precise prediction of the ground truth

Multiple vehicle fusion for a robust road condition estimation based on vehicle sensors and data mining

The road condition is an important factor for driving comfort and has impact on safety, economy and health. Delayed detection of defects lead to renewals which yields to complete roadblocks or traffic jams. Therefore, an early identification of road defects is desirable. Novel condition monitoring systems employ vehicles as sensor platforms and apply machine learning methods to predict the road condition based on the sensor data. The paper addresses the question how to combine the classification results from different vehicles to improve the final prediction. Different fusion strategies are investigated in various scenarios in a novel simulation. It is demonstrated that the performance of the classification can be improved compared to a majority vote or only considering one vehicle by taking the probability for the prediction of each vehicle into account. The probabilities follow a multinomial distribution and the precision matrix of the classifiers provide the best parameters. Overall, the results show that the application of the presented fusion strategies on road condition estimation greatly improve the performance and guarantee a robust detection of defects

Multiple vehicle fusion for a robust road condition estimation based on vehicle sensors and data mining

The road condition is an important factor for driving comfort and has impact on safety, economy and health. Delayed detection of defects lead to renewals which yields to complete roadblocks or traffic jams. Therefore, an early identification of road defects is desirable. Novel condition monitoring systems employ vehicles as sensor platforms and apply machine learning methods to predict the road condition based on the sensor data. The paper addresses the question how to combine the classification results from different vehicles to improve the final prediction. Different fusion strategies are investigated in various scenarios in a novel simulation. It is demonstrated that the performance of the classification can be improved compared to a majority vote or only considering one vehicle by taking the probability for the prediction of each vehicle into account. The probabilities follow a multinomial distribution and the precision matrix of the classifiers provide the best parameters. Overall, the results show that the application of the presented fusion strategies on road condition estimation greatly improve the performance and guarantee a robust detection of defects

Characterization of Road Condition with Data Mining Based on Measured Kinematic Vehicle Parameters

This work aims at classifying the road condition with data mining methods using simple acceleration sensors and gyroscopes installed in vehicles. Two classifiers are developed with a support vector machine (SVM) to distinguish between different types of road surfaces, such as asphalt and concrete, and obstacles, such as potholes or railway crossings. From the sensor signals, frequency-based features are extracted, evaluated automatically with MANOVA. The selected features and their meaning to predict the classes are discussed. The best features are used for designing the classifiers. Finally, the methods, which are developed and applied in this work, are implemented in a Matlab toolbox with a graphical user interface. The toolbox visualizes the classification results on maps, thus enabling manual verification of the results. The accuracy of the cross-validation of classifying obstacles yields 81.0% on average and of classifying road material 96.1% on average. The results are discussed on a comprehensive exemplary data set. SCOPUS: ar.j info:eu-repo/semantics/published Document type: Articl